Granger Causality for Compressively Sensed Sparse Signals

Compressed sensing is a scheme that allows for sparse signals to be acquired, transmitted and stored using far fewer measurements than done by conventional means employing Nyquist sampling theorem. Since many naturally occurring signals are sparse (in some domain), compressed sensing has rapidly seen popularity in a number of applied physics and engineering applications, particularly in designing signal and image acquisition strategies, e.g., magnetic resonance imaging, quantum state tomography, scanning tunneling microscopy, analog to digital conversion technologies. Contemporaneously, causal inference has become an important tool for the analysis and understanding of processes and their interactions in many disciplines of science, especially those dealing with complex systems. Direct causal analysis for compressively sensed data is required to avoid the task of reconstructing the compressed data. Also, for some sparse signals, such as for sparse temporal data, it may be difficult to discover causal relations directly using available data-driven/ model-free causality estimation techniques. In this work, we provide a mathematical proof that structured compressed sensing matrices, specifically Circulant and Toeplitz, preserve causal relationships in the compressed signal domain, as measured by Granger Causality. We then verify this theorem on a number of bivariate and multivariate coupled sparse signal simulations which are compressed using these matrices. We also demonstrate a real world application of network causal connectivity estimation from sparse neural spike train recordings from rat prefrontal cortex.Comment: Submitted to IEEE Transactions on Neural Networks and Learning System

Theoretical and Experimental Investigations into Causality, its Measures and Applications

A major part of human scientific endeavour aims at making causal inferences of observed phenomena. While some of the studies conducted are experimental, others are observational, the latter often making use of recorded data. Since temporal data can be easily acquired and stored in today’s world, time-series causality estimation measures have come into wide use across a range of disciplines such as neuroscience, earth science and econometrics. In this context, model-free/data-driven methods for causality estimation are extremely useful, as the underlying model generating the data is often unknown. However, existing data-driven measures such as Granger Causality and Transfer Entropy impose strong statistical assumptions on the data and can only estimate causality by associational means. Associational causality, being the most rudimentary level of causality has several limitations. In this thesis, we propose a novel Interventional Complexity Causality scheme for time-series measurements so as to capture a higher level of causality based on intervention which until now could be inferred only through model-based measures. Based on this interventional scheme, we formulate a Compression-Complexity Causality (CCC) measure that is rigorously tested on simulations of stochastic and deterministic systems and shown to overcome the limitations of existing measures. CCC is then applied to infer causal relations from real data mainly in the domain of neuroscience. These include the study of brain connectivity in human subjects performing a motor task and a study to distinguish between awake and anaesthesia states in monkeys using electrophysiological brain recordings. Through theoretical and empirical advances in causality testing, the thesis also makes contributions to a number of allied disciplines. A causal perspective is given for the ubiquitous phenomenon of chaotic synchronization. One of the major contributions in this regard is the introduction of the notion of Causal Stability and formulation (with proof) of a novel Causal Stability Synchronization Theorem which gives a condition for complete synchronization of coupled chaotic systems. Further, we propose and test for techniques to analyse causality between sparse signals using compressed sensing. A real application is demonstrated for the case of sparse neuronal spike trains recorded from rat prefrontal cortex. The area of temporal-reversibility detection of time-series is also closely linked to the domain of causality testing. We develop and test a new method to check for time-reversibility of processes and explore the behaviour of causality measures on coupled time-reversed processes

Time-Reversibility, Causality and Compression-Complexity

Detection of the temporal reversibility of a given process is an interesting time series analysis scheme that enables the useful characterisation of processes and offers an insight into the underlying processes generating the time series. Reversibility detection measures have been widely employed in the study of ecological, epidemiological and physiological time series. Further, the time reversal of given data provides a promising tool for analysis of causality measures as well as studying the causal properties of processes. In this work, the recently proposed Compression-Complexity Causality (CCC) measure (by the authors) is shown to be free of the assumption that the "cause precedes the effect", making it a promising tool for causal analysis of reversible processes. CCC is a data-driven interventional measure of causality (second rung on the Ladder of Causation) that is based on Effort-to-Compress (ETC), a well-established robust method to characterize the complexity of time series for analysis and classification. For the detection of the temporal reversibility of processes, we propose a novel measure called the Compressive Potential based Asymmetry Measure. This asymmetry measure compares the probability of the occurrence of patterns at different scales between the forward-time and time-reversed process using ETC. We test the performance of the measure on a number of simulated processes and demonstrate its effectiveness in determining the asymmetry of real-world time series of sunspot numbers, digits of the transcedental number π and heart interbeat interval variability

Causal stability and synchronization

Synchronization of chaos arises between coupled dynamical systems and is very well understood as a temporal phenomenon, which leads the coupled systems to converge or develop a dependence with time. In this work, we provide a complementary spatial perspective to this phenomenon by introducing the novel idea of causal stability. We then propose and prove a causal stability synchronization theorem as a necessary and sufficient condition for complete synchronization. We also provide an empirical criterion to identify synchronizing variables in coupled identical chaotic dynamical systems based on intrasystem causal influences estimated using time series data of the driving system alone. For this, a recently proposed measure, Compression-Complexity Causality (CCC), is used. The sign and magnitude of the estimated CCC value capture the nature of dynamical influences from each variable to rest of the subsystem and are thus able to determine whether or not the variable, when used to couple another system, will drive that system to synchronization. ACKNOWLEDG

Data-based intervention approach for Complexity-Causality measure

Causality testing methods are being widely used in various disciplines of science. Model-free methods for causality estimation are very useful, as the underlying model generating the data is often unknown. However, existing model-free/data-driven measures assume separability of cause and effect at the level of individual samples of measurements and unlike model-based methods do not perform any intervention to learn causal relationships. These measures can thus only capture causality which is by the associational occurrence of ‘cause’ and ‘effect’ between well separated samples. In real-world processes, often ‘cause’ and ‘effect’ are inherently inseparable or become inseparable in the acquired measurements. We propose a novel measure that uses an adaptive interventional scheme to capture causality which is not merely associational. The scheme is based on characterizing complexities associated with the dynamical evolution of processes on short windows of measurements. The formulated measure, Compression-Complexity Causality is rigorously tested on simulated and real datasets and its performance is compared with that of existing measures such as Granger Causality and Transfer Entropy. The proposed measure is robust to the presence of noise, long-term memory, filtering and decimation, low temporal resolution (including aliasing), non-uniform sampling, finite length signals and presence of common driving variables. Our measure outperforms existing state-of-the-art measures, establishing itself as an effective tool for causality testing in real world applications

ChaosNet: A Chaos Based Artificial Neural Network Architecture for Classification

Inspired by chaotic firing of neurons in the brain, we propose ChaosNet—a novel chaos based artificial neural network architecture for classification tasks. ChaosNet is built using layers of neurons, each of which is a 1D chaotic map known as the Generalized Luröth Series (GLS) that has been shown in earlier works to possess very useful properties for compression, cryptography, and for computing XOR and other logical operations. In this work, we design a novel learning algorithm on ChaosNet that exploits the topological transitivity property of the chaotic GLS neurons. The proposed learning algorithm gives consistently good performance accuracy in a number of classification tasks on well known publicly available datasets with very limited training samples. Even with as low as seven (or fewer) training samples/class (which accounts for less than 0.05% of the total available data), ChaosNet yields performance accuracies in the range of 73.89%−98.33%. We demonstrate the robustness of ChaosNet to additive parameter noise and also provide an example implementation of a two layer ChaosNet for enhancing classification accuracy. We envisage the development of several other novel learning algorithms on ChaosNet in the near future